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Program Description 1 


Accelerated Math, published by Renaissance Learning, is a 
software tool used to customize assignments and monitor 
progress in math for students in grades 1-12. The Accelerated 
Math software creates individualized assignments aligned with 
state standards and national guidelines, scores student work, 
and generates reports on student progress. The software can 


be used in conjunction with the existing math curriculum to add 
practice components and potentially aid teachers in differentiat- 
ing instruction through the program’s progress-monitoring data. 
Studies in this review assess the effectiveness of Accelerated 
Math as part of a school’s core math curriculum. 


Research 


No studies of Accelerated Math meet What Works Clearinghouse 
(WWC) evidence standards, and three studies meet WWC 
evidence standards with reservations. These three studies, 
which included approximately 2,200 middle school students 
in grades 6-8, compared standardized test scores of students 


who used Accelerated Math with those of students who used 
traditional curricula. 2 

Based on these three studies, the WWC considers the extent 
of evidence for Accelerated Math to be medium to large for 
math achievement. 



Effectiveness Accelerated Math was found to have no discernible effects on math achievement. 



Math Achievement 
Rating of effectiveness No discernible effects 

Improvement index 3 Average: +4 percentile points 

Range: -3 to +7 percentile points 

1. The descriptive information for this program was obtained from publicly-available sources: the program’s website ( http://www.renlearn.com/am/ . down- 
loaded July 2008), Nunnery and Ross (2007), Ysseldyke and Bolt (2007), and Ysseldyke and Tardrew (2007). The WWC requests developers to review the 
program description sections for accuracy from their perspective. Further verification of the accuracy of the descriptive information for this program is 
beyond the scope of this review. 

2. The evidence in this report is based on available research. Findings and conclusions may change as new research becomes available. 

3. These numbers show the average and range of student-level improvement indices for all findings across the studies. 
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Additional program Developer and contact 

information Renaissance Learning developed and distributes Accelerated 

Math. Address: PO Box 8036, Wisconsin Rapids, Wl 54495-8036. 
Email: answers@renlearn.com . Web: www.renlearn.com/am/ . 

Telephone: (800) 338-4204. 

Scope of use 

Accelerated Math was first released in 1998. In 2008 Renais- 
sance Learning released the Accelerated Math Second-Edition 
Libraries, which included a revised scope and sequence for 
grades 1-8, algebra I, and geometry. According to the develop- 
ers, more than 30,000 schools nationwide us e Accelerated Math 
and other Renaissance Learning math programs. 

Teaching 

The Accelerated Math software can be used with existing 
textbooks and instructional methods for students in grades 1 
through high school to add practice assignments and progress 
monitoring to the existing curriculum. Students are placed into 
grade-level libraries in Accelerated Math based on teacher 
discretion or their performance on a norm-referenced, standard- 
ized measure of general math achievement. After instruction 
on a math objective, teachers can use the software to create 



individualized practice assignments for students. Students 
then record their answers through a handheld responder or on 
forms that are scanned into the computer. After scoring the 
assignment, the software generates a report showing student 
progress in mastering the objective as well as information about 
items answered correctly and incorrectly. Teachers also receive 
student- and classroom-level reports. After reviewing students’ 
progress, teachers can adjust instruction for the entire class, for 
small groups of students struggling with similar objectives, or 
for individual students as needed. Accelerated Math generates 
future assignments based on a student’s performance on previ- 
ous assignments. 

Cost 

The Accelerated Math Enterprise Edition is available for a $2,899 
one-time school fee, plus a $1,000 annual fee for up to 250 
students. Additional students cost $4 each per year. The Enterprise 
license includes nine hours of web-based professional develop- 
ment, content libraries for grade 1 math through calculus, unlimited 
technical support, software updates, and hosting of the software 
for the first year. After the first year, web hosting costs $399 a year. 
Single classroom packages are also available. The cost of an opti- 
cal scanner (needed to grade student assignments) is not included. 



Research Thirty-eight studies reviewed by the WWC investigated the 

effects of Accelerated Math. None are randomized controlled 
trials that meet WWC evidence standards. Three studies (Nun- 
nery & Ross, 2007; Ysseldyke & Bolt, 2007; Ysseldyke & Tardrew, 
2007) are randomized controlled trials or quasi-experimental 
designs that meet WWC evidence standards with reservations. 
The remaining 35 studies do not meet either WWC evidence 
standards or eligibility screens. 

Meets evidence standards 

No studies meet evidence standards. 

Meets evidence standards with reservations 

Nunnery and Ross (2007) conducted a quasi-experiment to 
assess the impact of the School Renaissance program— a 



comprehensive school reform model, which includes the Acceler- 
ated Math program— on the math achievement of students in a 
suburban Texas school district. Treatment schools implemented 
the program. Although supplemented by a professional develop- 
ment component known as Math Renaissance, the program’s 
key math component was Accelerated Math. Math achievement 
was measured by the Texas Learning Index math scores obtained 
from the Texas Assessment of Academic Skills. Two treatment 
middle schools were matched to two comparison middle schools 
based on the Texas Education Association’s Academic Excel- 
lence Indicator System (AEIS). The AEIS groups each school 
with 40 similar schools based on their percentage of African- 
American, Hispanic, White, economically disadvantaged, and 
limited English proficient students as well as student mobility 
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Research (continued) rates as determined by cumulative attendance. From a list of 
40 similar schools, the most similar school was matched to the 
treatment school, with preference given to those schools that did 
not implement Accelerated Math or other components of School 
Renaissance. The authors did not describe the existing math 
curriculum in the treatment or comparison schools. Although the 
study sample included students in grades 3-8, only students 
in grades 6-8 are relevant to this review. The analysis sample 
included 992 students in four middle schools (482 students in 
two treatment schools and 510 in two comparison schools) in 
grades 6-8. The findings section reports the effectiveness of the 
Accelerated Math program for the grade 6-8 cohort. 

Ysseldyke and Bolt (2007) conducted a randomized controlled 
trial with severe attrition. The authors randomly assigned class- 
rooms to treatment and control groups to assess the impact 
of Accelerated Math on the STAR Math and Terra Nova exams. 
Principals who had shown interest in Accelerated Math were 
contacted to participate in the study. Ultimately, Accelerated Math 
was implemented in eight schools in seven districts in seven states 
(two schools in Texas and one each in Alabama, Florida, Michigan, 
Mississippi, North Carolina, and South Carolina). The study sample 
included students in grades 2-8, but only those in grades 6-8 
are relevant to this review. The middle school analysis sample 
included more than 450 students in 21 treatment classrooms and 
approximately 400 students in 19 control classrooms. In middle 
schools, intact classrooms were randomly assigned to treatment 
and control groups. Because middle school math teachers taught 
multiple classes, study teachers taught both Accelerated Math 
classes (the treatment condition) and traditional classes. Treat- 
ment classrooms were assigned to be taught using Accelerated 
Math as an integrated addition to the existing math curriculum. 
Control classrooms were assigned to be taught using the existing 
curriculum without Accelerated Math. In practice, the Accelerated 
Math program was not implemented for approximately 40% of 



students in grades 2-8 in the initial treatment sample; the authors 
did not report the implementation percentage for the middle school 
analysis sample. The study meets standards with reservations 
because of a severe overall attrition rate. 

Ysseldyke and Tardrew (2007) conducted a classroom 
matched-pairs quasi-experimental design to assess Accelerated 
Math’s impact on posttest scores on the STAR Math test. The 
study was designed for school principals to randomly assign 
classrooms to treatment or comparison conditions; however, the 
authors had no control over this process and reported that they 
had no basis for claiming that random assignment occurred. 
Thus, the WWC reviewed the study as a quasi-experimental 
design. The total study included 2,397 students in 125 class- 
rooms in 27 schools in 24 states (Alabama, Arkansas, California, 
Georgia, Idaho, Illinois, Indiana, Iowa, Kansas, Massachusetts, 
Michigan, Minnesota, Missouri, Montana, New Mexico, Ohio, 
Oklahoma, Oregon, Pennsylvania, Tennessee, Texas, Virginia, 
Washington, and Wisconsin). Results are reported by grade for 
grades 3-6 and in cohorts for students in grades 7-8 and 9-10. 
The grade 6 sample included 326 students in 17 classrooms 
(169 students in nine treatment classrooms and 157 students in 
eight comparison classrooms). The grade 7-8 sample included 
149 students in four classrooms (66 students in two treatment 
classrooms and 83 students in two comparison classrooms). 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or medium to large (see the What Works Clearinghouse 
Extent of Evidence Categorization Scheme) . The extent of 
evidence takes into account the number of studies and the 
total sample size across the studies that meet WWC evidence 
standards, with or without reservations. 4 

The WWC considers the extent of evidence for Accelerated 
Math to be medium to large for math achievement. 



4. The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the student demographics and the settings 
in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was determined for 
Accelerated Math is in Appendix A5. 
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Effectiveness 



Findings 

The WWC review of interventions for middle school math 
addresses student outcomes in the math achievement domain. 

Nunnery and Ross (2007) reported a positive and statistically 
significant effect of Accelerated Math on overall math achieve- 
ment based on the Texas Learning Index math scores. After 
accounting for the misalignment between the school as the unit 
of assignment and the student as the unit of analysis, the WWC 
determined that this finding was neither statistically significant 
nor substantively important according to WWC criteria (an effect 
size greater than 0.25). 5 

Ysseldyke and Bolt (2007) examined two outcomes in this 
domain: the STAR Math test and Terra Nova math subtest. The 
authors reported a statistically significant positive effect for one 
outcome (STAR Math) and no statistically significant effect for the 
other (Terra Nova). 6 After adjusting for misalignment between the 
classroom as the unit of assignment and the student as the unit 
of analysis, the WWC determined that, for both outcomes, the 
effects were neither statistically significant nor large enough to be 
considered substantively important according to WWC criteria. 7 

Ysseldyke and Tardrew (2007) reported a positive and statisti- 
cally significant effect of Accelerated Math for the grade 6 



classrooms on overall math achievement based on STAR Math 
scores. They also reported a positive, but not statistically sig- 
nificant, effect for the grade 7-8 Accelerated Math classrooms. 
After adjusting for misalignment between the classroom as the 
unit of assignment and the student as the unit of analysis, the 
WWC determined that neither finding was statistically significant 
nor large enough to be considered substantively important 
according to WWC criteria. 8 

In sum, in the math achievement domain the WWC reviewed 
findings from four samples reported in three studies. 9 All four 
samples showed indeterminate effects. No studies implemented 
a strong design. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme) . 



The WWC found 
Accelerated Math to have 
no discernible effects for 
math achievement 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see echnical Details 

of WWC-Conducted Computations) . The improvement index rep- 
resents the difference between the percentile rank of the average 



student in the intervention condition versus the percentile rank of 
the average student in the comparison condition. Unlike the rating 
of effectiveness, the improvement index is based entirely on the 
size of the effect, regardless of the statistical significance of the 
effect, the study design, or the analyses. The improvement index 
can take on values between -50 and +50, with positive numbers 
denoting results favorable to the intervention group. 



5. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . For the formulas the WWC used to calculate 
the statistical significance, see Technical Details of WWC-Conducted Computations . In the case of Nunnery and Ross (2007), a correction for clustering 
was needed, so the significance levels may differ from those reported in the original study. 

6. The study authors provided the WWC with findings for the WWC-relevant grade levels. 

7. In the case of Ysseldyke and Bolt (2007), a correction for clustering was needed, so the significance levels may differ from those reported in the original study. 

8. In the case of Ysseldyke and Tardrew (2007), a correction for clustering was needed. 

9. The two grade-level cohorts— grade 6 and grades 7-8— in the Ysseldyke and Tardrew (2007) study were treated as separate studies because they 
examined the effects of Accelerated Math on different samples of students. 
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The WWC found 
Accelerated Math to have 
no discernible effects for 
math achievement 

(continued) 



The average improvement index for math achievement is +4 
percentile points across the four study samples in the three stud- 
ies, with a range of -3 to +7 percentile points across findings. 

Summary 

The WWC reviewed 38 studies on Accelerated Math. None meet 
WWC evidence standards; three studies meet WWC evidence 



standards with reservations; the remaining 35 studies do not 
meet either WWC evidence standards or eligibility screens. 
Based on the three studies, the WWC found no discernible 
effects in math achievement. The conclusions presented in this 
report may change as new research emerges. 
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For more information about specific studies and WWC calculations, please see the WWC Accelerated Math 
Technical Appendices . 
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Appendix 



Appendix A1.1 Study Characteristics: Nunnery & Ross, 2007 (quasi-experimental design) 



Characteristic 


Description 


Study citation 


Nunnery, J. A., & Ross, S. M. (2007). The effects of the School Renaissance program on student achievement in reading and mathematics. Research in the Schools, 
74(1), 40-59. 


Participants 


The analysis sample included 992 students (482 treatment, 510 comparison) in the 2001/02 grade 8 cohort from four middle schools (two treatment and two comparison). 
Of the student sample, 21% qualified for free or reduced-price lunch (21% treatment, 20% comparison), 4% were limited English proficient (4% treatment, 4% comparison), 
7% African-American (9% treatment, 6% comparison), 3% Asian (2% treatment, 3% comparison), 19% Hispanic (19% treatment, 20% comparison), 0% Native American 
(0% treatment, 0% comparison), and 70% were White (69% treatment, 71% comparison). Information about attrition was provided only at the level of assignment. Of the 
11 elementary and middle schools originally selected as comparison schools, three schools did not provide appropriate grade-level test score data and were replaced (it is 
unknown whether any of these replaced schools were middle schools). Students in the analysis sample remained in the same school and had matched data available for 
three consecutive years (1999/2000-2001/02). 


Setting 


The treatment group schools came from one suburban school district in Texas. Comparison schools came from other school districts in Texas with similar populations of students. 


Intervention 


In 2000/01 , schools in the treatment group implemented School Renaissance, a comprehensive school reform model that includes Accelerated Math. Accelerated Math is a 
progress-monitoring software program that tracks students' daily activities, provides immediate feedback to students and teachers, alerts teachers to students struggling with 
certain assignments, and monitors achievement. Teachers can use the program with their existing math curriculum. Students in the treatment group experienced two years 
of the Accelerated Math program. 


Comparison 


Schools in the comparison condition were from Texas school districts that had not implemented the full School Renaissance package. However, it is still possible that some 
elements of Accelerated Math were present in the comparison schools. 


Primary outcomes 
and measurement 


The study used the Texas Learning Index (TLI) math scores (based on the Texas Assessment of Academic Skills); for the grade 8 cohort, program comparisons were based on 
average transformed scores for grades 7 and 8 from 2001 and 2002. The TLI has a common interpretation across grades: a score of 70 or above indicates performance 
at or above grade-level expectations. A student receiving the same score at consecutive grade levels made one year of academic progress. Lor a more detailed description 
of these outcome measures, see Appendix A2. 


Staff/teacher training 


A Renaissance coach conducts an initial training seminar and provides ongoing assistance to teachers. 
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Appendix A1.2 Study Characteristics: Ysseldyke & Bolt, 2007 (randomized controlled trial with attrition) 



Characteristic 


Description 


Study citation 


Ysseldyke, J., & Bolt, D. M. (2007). Effect of technology-enhanced continuous progress monitoring on math achievement. School Psychology Review, 36(3), 453-467. 


Participants 1 


The initial study sample included 3,309 students in grades 2-8 during the 2003/04 school year from 133 classrooms in nine schools, representing eight school districts in 
eight states. In the initial study sample 1% of the students were Asian, 28% African-American, 38% Hispanic, 0% Native American, 24% White, and 8% not specified. This 
review focuses on the middle school sample, which initially included 1,823 grade 6-8 students (1,010 treatment and 813 control) in 73 classrooms (41 treatment and 32 
control). Demographic data on the middle school students could not be culled from the original study. Middle school classrooms dropped from the analysis include: 7 special 
education or enrichment treatment classrooms taught by teachers who had access to, but did not receive training in, Accelerated Math ; 4 classrooms (2 treatment, 2 control) 
taught by two teachers who, according to the authors, arbitrarily chose which students to treat; and 22 classrooms (11 treatment, 11 controls) in a large, urban middle school 
district that, according to the authors, was unable to devote sufficient time and resources to Accelerated Math. The results here are drawn from the test-takers in the 40 
middle school classrooms (21 treatment, 19 control) included in the analysis — 792 students took the STAR Math test (418 treatment, 374 control) and 851 took the Terra 
Nova test (454 treatment, 397 control). Postattrition treatment and control groups were equivalent on pretests at baseline. Because these samples reflect attrition rates 
greater than 20%, the WWC rated this study as meeting evidence standards with reservations. 


Setting 


The study took place in eight schools in seven districts in seven states: Alabama, Florida, Michigan, Mississippi, North Carolina, South Carolina, and two schools in Texas. 
The middle school sample analyzed here comprises three schools in Michigan, Mississippi, and North Carolina. 


Intervention 


Students were taught by teachers using the Accelerated Math program during the 2003/04 school year. Accelerated Math is a progress-monitoring software program that 
teachers can use with their existing math curriculum. The program tracks students’ daily activities, provides immediate feedback to students and teachers, alerts teachers 
to students struggling with certain assignments, and monitors student achievement. Teachers assigned to the treatment group were asked to use Accelerated Math with their 
present math curriculum. In practice, the program was not implemented for approximately 40% of grade 2-8 students in the initial treatment group; the authors did not report 
the percentage of grade 6-8 students in the treatment group of the analysis sample that did not participate in Accelerated Math. 


Comparison 


Students in the control group were taught using existing math curricula, without Accelerated Math. The existing curricula included: Scott Foresman Middle School Math, 
Consumer Math, Everyday Math, Transition Math (Prentice Hall), and Chicago Math in Michigan; Glencoe in Mississippi; and Glencoe, McGraw-Hill, and the state curriculum 
in North Carolina. Control students had the same teachers as the intervention group students. 


Primary outcomes 
and measurement 


Participating students were pretested in October 2003 and posttested in May 2004 using two nationally normed, standardized tests (STAR Math and Terra Nova) for math 
achievement. Students in the treatment and control groups were compared using a linear regression analysis in which posttest scores were regressed on pretest scores 
and on dummy variables related to main effects for experimental condition and school. 


Staff/teacher training 


Teachers in the intervention group were trained to use Accelerated Math. During the school year, teachers using Accelerated Math received three to five visits from a Renaissance 
Learning math consultant, who guided teachers on how to improve their use of the program. Teachers also had unlimited access to technical support. 



1. The study authors provided the WWC with sample sizes for the middle schools. 
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Appendix A1.3 Study Characteristics: Ysseldyke & Tardrew, 2007 (quasi-experimental design) 



Characteristic 


Description 


Study citation 


Ysseldyke, J, & Tardrew, S. (2007). Use of a progress monitoring system to enable teachers to differentiate mathematics instruction. Journal of Applied School Psychology, 
24(1), 1-28. 


Participants 1 


The initial study sample included 2,397 students (1,319 treatment and 1,078 comparison) in grades 3-10 during the 2001/02 school year from 125 classrooms (67 treatment 
and 58 comparison) in 47 schools in 24 states. The middle school analysis sample in this review included 475 grade 6-8 students (235 treatment, 240 comparison) in 25 
classrooms (13 treatment, 12 comparison). Of the students, 43% were male (43% treatment, 43% comparison), and 49% female (48% treatment, 51% comparison). Of 
the total student gender, 7% were reported as unspecified (8% treatment, 6% comparison). Of the students, 0% were Asian (0% treatment, 0% comparison), 1% African- 
American (1% treatment, 0% comparison), 9% Hispanic (9% treatment, 9% comparison), 0% Native American (0% treatment, 0% comparison), 44% White (38% treatment, 
49% comparison), and 46% were reported as unspecified (51% treatment, 42% comparison). 


Setting 


The study was conducted in 47 schools in 24 states (Alabama, Arkansas, California, Georgia, Idaho, Illinois, Indiana, Iowa, Kansas, Massachusetts, Michigan, Minnesota, 
Missouri, Montana, New Mexico, Ohio, Oklahoma, Oregon, Pennsylvania, Tennessee, Texas, Virginia, Washington, and Wisconsin). The authors did not report whether all 
schools and states were represented in the middle school (grades 6-8) sample. 


Intervention 


Students were taught by teachers using Accelerated Math during the spring semester of the 2001/02 school year. A progress-monitoring software program, Accelerated Math 
can be used with teachers’ existing math curriculum. The program tracks students’ daily activities, provides immediate feedback to students and teachers, alerts teachers' to 
students struggling with certain assignments, and monitors student achievement. Teachers assigned to the Accelerated Math treatment group were asked to use the program 
with their existing math curriculum. 


Comparison 


Teachers assigned to the comparison group did not use Accelerated Math but continued their usual math curriculum and practices. 


Primary outcomes 
and measurement 


Using a computer adaptive test of math achievement (STAR Math), students were pretested in January 2002 and posttested in May 2002. 


Staff/teacher training 


Intervention teachers participated in a one-day training session conducted by Renaissance Learning. The training was designed to familiarize teachers with Accelerated Math 
and to guide them in integrating it into curriculum and instruction. Of 68 treatment group teachers in the full sample, 66 attended the training. 



1. The study authors provided the WWC with the number of middle school classrooms by treatment status. 
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Appendix A2 Outcome measures for the math achievement domain 



Outcome measure 


Description 


Texas Learning index 
math scores (based on 
the Texas Assessment 
of Academic Skills) 


The Texas Assessment of Academic Skills (TAAS) is a criterion-referenced standardized state test that measures problem-solving and critical-thinking skills. The Texas Learn- 
ing Index (TLI) is an outcome metric, based on student performance on the TAAS, which allows for comparisons between administrations and between grades. The TLI has 
a common interpretation across grades: a score of 70 or above indicates the student performed at or above grade-level expectations. A student receiving the same score at 
consecutive grade levels made one year of academic progress. Analyses in the study were based on a transformation of the TLI that was conducted to induce normality. 


STAR Math assessment 


STAR Math is a computer-adaptive math test that assesses math skills, it combines computation and numeration items with word problems, estimation, statistics, charts and 
graphs, geometry, measurement, and algebra. STAR scores can appear as scaled scores or normal curve equivalent values. 


Terra Nova 
mathematics subtest 


The Terra Nova subtest is a national norm-referenced test that assesses academic performance in math. 
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Appendix A3 Summary of study findings included in the rating for the math achievement domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(clusters/ 
students) 


Accelerated Math Comparison 

group group 


Mean difference 3 
(Accelerated 
Math- 

comparison) 


Statistical 

Effect significance 5 

size 4 (at a = 0.05) 


Improvement 

index 6 



2001 and 2002 transformed 
Texas Learning Index scores 


Grade 8 cohort 


Nunnery & Ross, 2007 (quasi-experimental design) 7 

4/992 1.21 s 1.16 s 0.05 

(0.47) (0.44) 


0.11 


ns 


+4 


Average for math achievement (Nunnery & Ross, 2007) 9 








0.11 


ns 


+4 






Ysseldyke & Bolt, 2007 (randomized controlled trial with attrition) 7 








2004 STAR Math normal 


Grades 6-8 


40/792 


48.1 1 10 


44.45 11 


3.66 


0.19 


ns 


+7 


curve equivalent scores 






(18.90) 


(20.06) 










2004 Terra Nova normal 


Grades 6-8 


40/851 


46.89 10 


48.17 11 


-1.28 


-0.07 


ns 


-3 


curve equivalent scores 






(18.67) 


(18.69) 










Average for math achievement (Ysseldyke & Bolt, 2007) 9 








0.06 


ns 


+2 


Ysseldyke & Tardrew, 2007, grade 6 cohort (quasi-experimental design) 7 


2002 STAR Math 


Grade 6 


17/326 


773.43 12 


762.80 13 


10.63 


0.10 


ns 


+4 


scale scores 






(114.49) 


(93.82) 










Average for math achievement (Ysseldyke & Tardrew, 2007 grade 6) 9 








0.10 


ns 


+4 


Ysseldyke & Tardrew, 2007, grades 7 and 8 cohort (quasi-experimental design) 7 


2002 STAR Math 


Grades 


8/149 


801.14 12 


786.47 13 


14.67 


0.17 


ns 


+7 


scale scores 


7 and 8 




(87.53) 


(83.33) 










Average for math achievement (Ysseldyke & Tardrew, 2007, grades 7 and 8) 9 






0.17 


ns 


+7 


Domain average for math achievement across all studies 9 








0.11 


na 


+4 



ns = not statistically significant 
na = not applicable 



1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the math achievement domain. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

4. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 



(continued) 
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Appendix A3 Summary of study findings included in the rating for the math achievement domain (continued) 



6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations . In the case of Nunnery and Ross (2007), a correction for clustering was needed, so the significance levels may differ from those reported in the original study; 
no corrections for multiple comparisons were needed because there is only one outcome in this domain. In the case of Ysseldyke and Bolt (2007), a correction for clustering was needed, so 

the significance levels may differ from those reported in the original study; no corrections for multiple comparisons were needed because the WWC-computed effect sizes were not statistically 
significant. In the case of Ysseldyke and Tardrew (2007), a correction for clustering was needed, so the significance levels may differ from those reported in the original study; no corrections for 
multiple comparisons were needed because there is only one outcome in this domain. 

8. Nunnery and Ross (2007, pp. 45-46) computed a transformation of the Texas Learning Index score to induce the distribution into normality to allow for an analysis of covariance. 

9. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated 
from the average effect sizes. 

10. The intervention group values from Ysseldyke and Bolt (2007) are the control group average plus the program coefficient from a regression analysis that controls for baseline pretest scores. The 
study authors provided the WWC with the program coefficient, unadjusted average, and standard deviations for both groups. 

11. The control group average from Ysseldyke and Bolt (2007) are unadjusted. 

12. The intervention group values from Ysseldyke and Tardrew (2007) are the comparison group means plus the difference in mean gains between the intervention ( Accelerated Math) and compari- 
son groups. 

13. The comparison group means from Ysseldyke and Tardrew (2007) are unadjusted. 
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Appendix A4 Accelerated Math rating for the math achievement domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of math achievement, the WWC rated Accelerated Math as no discernible effects. The remaining ratings (potentially negative effects and 
negative effects) were not considered, as Accelerated Math was assigned the highest applicable rating. 



Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed statistically significant or substantively important positive or negative effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statistically significant positive effects. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed statistically significant or substantively important negative effects. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed statistically significant or substantively important positive effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. The three studies that evaluated math achievement and met WWC standards showed indeterminate effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through EITHER of the following. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed statistically significant or substantively important effects, either positive or negative. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed statistically significant or substantively important effects, either positive or negative. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings 
of potentially positive or potentially negative effects. For a complete description, see the WWC Intervention Rating Scheme . 
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Appendix A5 Extent of evidence by domain 



Outcome domain 


Number of studies 


Schools 


Sample size 


Students 


Extent of evidence 1 


Math achievement 


3 


>7 2 




>2,259 3 


medium to large 



1. A rating of “medium to large” requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. Other- 
wise, the rating is “small.” 

2. Nunnery and Ross (2007) include four middle schools. Ysseldyke and Bolt (2007) include three middle schools. Ysseldyke and Tardrew (2007) do not report the number of middle schools. 

3. Nunnery and Ross (2007) include 992 middle school students in the analysis sample. Ysseldyke and Tardrew (2007) include 326 grade 6 students and 149 grade 7 and 8 students. Ysseldyke 
and Bolt (2007) include 792 students in the analysis of the STAR Math outcome and 851 students in the analysis of the Terra Nova outcome, but the authors do not report the extent of overlap 
between the two samples. 
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